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ABSTRACT 

Increased emphasis on public school accountability 
has resulted in several states including a science component in their 
elementary and middle school testing program. This component often 
includes items designed to measure process skill ability. The purpose 
of this study was to examine the performance of high school students 
on four types of multiple choice items used to assess student's 
ability to identify manipulated and responding variables. Items 
differed only in the stimulus material presented to students. 
Stimulus material was either a question focusing on the r.elationship 
between two variables, a hypothesis, a description of an experiment, 
or a description of results of an experiment. Student performance on 
these item types was compared to their performance on a standard 
Piagetian interview task of variable identification. The results of 
the study included: (1) the "question" format was the most difficult, 
while the "results" format appeared to be the easiest; (2) the 
"description" item type had a higher correlation with the total 
interview than any other item type. Among conclusions reached in this 
study was that although all four item types operate similarly, they 
did not correlate very highly with the interview/observation task. 
(CW) 
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Assessing the Ski*!! of Variable Identification 

with 

Multiple Choice Items 
Rationale 

The increased emphasis on accountability in public education 
has resulted in numerous states and local testing programs. 
Several states (e.g. Florida, South Carolina, Texas) have recently 
included a science component in the assessment of elementary and 
middle school students. This science component often includes 
items designed to measure process skill ability. Typically, 
process skill items are multiple choice format and do not include 
interaction with materials other than pencil and paper. Tests 
developed at the state and local level parallel those developed by 
science educators (e.g. Dillashaw & Okey, 1980; Cronin & Padilla, 
1986; McKenzie & Padilla, 1986). 

While the tests purport to measure process skill ability, it 
is interesting to note the form of the stimulus material. For 
example, in measuring the skill of variable identification the 
Test of Integrated Process Skills (TIPS, Dillashaw & Okey, 1980) 
and the Middle Grades Integrated Process Skills Test (MIPT, Cronin 
& Padilla, 1986) present a description of an experiment and ask 
the student to identify the manipulated (independent) or 
responding (dependent) variable. This approximates what may 
happen in the classroom when a teacher leads students in the 
planning and conducting of an experiment. However, it may be 
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argued that students might be successful identifying variables in 
a "concrete, hands-on" situation and fail to identify variables 
from the written description of an experiment, a more abstract 
situation. 

Testing. strategies employed by classroom teachers or district 
level personnel to measure process skill ability may take various 
forms. Teachers may assess their students' level of proficiency 
by using a checklist during a hands-on experiment or 
investigation. Another logical methods of assessment would be to 
use multiple choice items in which an experiment or investigation 
is described (as in TIPS and MIPT). While both of these methods 
of assessment may be employed, it is questionnable if they are of 
equal difficulty for students. 

Not only are we assessing process skill ability with both 
performance based observation and multiple choice items, but there 
exists a great deal of variability in the way multiple choice 
items are presented. For example, in a survey of multiple choice 
items used to measure the skill of variable identification, four 
different types were identified. Some items included a 
description of an experiment while others presented a written 
hypothesis, a science question, or the results of an experiment. 

Purpose 

The purpose of this study was to examine four types of 
multiple choice items used to assess students' ability to identify 
manipulated and responding variables. The items differed only in 
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the stimulus material presented to the student. Stimulus material 
was either a question, hypothesis, description of an experiment, 
or description of the results of an experiment. An example of 
each of the four item types examined follows: 
Question 

Sara wanted to find the answer to this question: "How does 
the number of batteries in a circuit affect the brightness of a 
bulb in the current?" If she answered this question by doing an 
experiment, what would be the responding variable? 

Hypothesis 

Sara conducted an experiment to test this hypothesis: "As the 
number of batteries in a circuit increases, the brightness of a 
bulb in the circuit increases." What is the responding variable 
in this experiment? 

Description of an Experiment 

Sara constructed a circuit made of one battery and one bulb. 
She noted the brightness of the bulb. She then added another 
battery to the circuit and noted the bulb's brightness. She 
continued this experiment by placing three and then four batteries 
in the circuit. What is the responding variable in this 
experiment? 

Description of the Results of an Experiment 

Sara conducted an experiment and found that as she increased 
the number of batteries in a circuit the brightness of the bulb in 
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the circuit increased. What is the responding variable in this 
experiment? 

In each of these four item types the same answer choices would 
be available to the students. For example: 

a. number of bulbs 

b. number of batteries 

c. bulb brightness 

d. battery strength 

In addition to examining subject performance across the four 
item types, performance on these items was compared to the 
subject's ability to identify manipulated and responding variables 
during a hands-on investigation. This involved interviewing each 
student, during two experimental situations, to ascertain their 
ability to identify manipulated and responding variables. The 
following research questions provided the focus of this study: 

1. Is there a difference in student performance across the 
four multiple choice items types? 

2. Is there a difference in the correlations between student 
scores on the four multiple choice item types and scores received 
during the hands-on investigation? 
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3. Is there a difference in student performance on items 
designed to measure the ability to identify manipulated variables 
and responding variables? 

Procedure and Data Analysis 
A sixteen item multiple choice test designed to measure the 
skill of variable identification was developed. Four items were 
written for each of the four item types previously described. 
Eight items (two per item type) require the subject to identify 
manipulated variables, while the remaining eight items (two per 
item type) require the identification of responding variables. 
Therefore, the total test could be broken into an eight item 
manipulated variable subtest and an eight item responding variable 
subtest. The test was then reviewed by a panel of educators. The 
panel, comprised of science educators, classroom teachers and 
scientists, provided comments on item appropriateness and clarity. 
Items were revised as indicated by the panel. A sample item is 
included as Figure 1. 

The variable identification test was administrated to a sample 
of high school students (n = 54). The sample was drawn from a 
high school in Southeast Louisiana in which process skills are 
included in the science curriculum. All subjects had been taught 
the terms manipulated variable and responding variable and had 
practiced identifying variables in experimental situations. 
Twenty five of the students were freshman with the remaining 
twenty nine sophomores. There were seventeen males in the study 
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and thirty seven females. Scores for this administration of the 
tests yielded a reliability estimate of .53 (KR-20). 

Each student was also observed and questioned while conducting 
two investigations. One investigation focused on identification 
of manipulated variables. Subjects were provided with a pendulum 
consisting of two different lengths of string and several bobs of 
different sizes and weights. Each subject was asked to identify 
variables that might affect the number of swings the pendulum 
would make in a given amount of time. The other investigation 
dealt with the identification of responding variables. Subjects 
were given a seltzer table to dissolve in a container 6f room 
temperature water. They were then given containers of warm and 
cold water and were asked to identify variables that might change 
or be different if seltzer tablets were dissolved in water of 
different temperatures. In an attempt to assure reliability of 
interviewers* a script was used in questioning the students during 
the investigations. The script used during one of the 
investigations is included as figure 2. Subjects were scored on 
their ability to identify variables while conducting each of the 
two investigations, using the following scheme. If a subject 
would not identify any variables, a score of 0 was assigned. 
Subjects identifying one variable were assigned a score of 1, two 
variables were assigned a score of two, while those identifying 
more than two variables were assigned a score of 3. 
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A testing scheme was used to control for the effects of the 
order on testing. Subjects were randomly assigned to one of four 
groups identified below: 

W M R 
W R M 
M R W 
R H W 

M = Written test 

M = Manipulated Variable Investigation 
R = Responding Variable Identi ficaiton 

Analysis of variance procedures revealed there were no 
statistically significant differences in the scores on either the 
written test or the investigations across these four groups. The 
four groups were collasped into one group for further analysis. 

Descriptive statistics and Spearman correlation coefficients 
were computed for all tests and subtests. Analysis of variance 
procedures were used to determine if difference existed on student 
scores across the four iteiTi types as well as between the 
manipulated variable and responding variable subtests. 

Results and Conclusions 
The following results based on the aforementioned research 
questions were obtained. 

1. Is there a difference in student's performance across the 
four multiple choice item types? 
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The subjects scored lowest on those items presented using the 
"question" format (1.72 out of a possible 4) (see Table 1). 
Scores were highest on items using the "results" format (2.46 out 
of a possible 4). There was no statistically significant 
differences in performance across the four item types. 

2. Is there a difference in the correlation between student 
^cores on the four item types and scores received during the 
hands-on investigation? 

The correlations between the four item types and the total 
investigation score ranged from -.06 to .24 (see Table 2). 
Furthermore the correlation between scores on the written 
manipulated and responding variable subtests and the hands-on 
investigating were also very low (-.09 to .15). This indicates 
that students might be classified differently on the basis of 
their ability to identify variables in these two testing 
situations. 

3. Is there a difference in student's performance on items 
designed to measure their ability to identify manipulated 
variables and responding variables? 

Although the mean scores on the written manipulated and 
written responding variable subtest were very similar (4.24 and 
4.09 respectively out of possible 8), the correlation between 
these two subtests was negative (r = -.05) (see Tables 1 and 2). 
A positive correlation was noted between the manipulated 
investigation task and the responding investigation task, hov/ever, 
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this correlation was low (r = .18). The means of the manipulated 
and responding interview tasks (2.24 and 1.69, respectively, out 
of a possible 3)^ were not significantly different. 

From this investigation the following conclusions appear 
tenable: 

1. Although all 4 item types operate similarly, they do not 
correlate highly with the interview/observation task. 

2. If a teacher only assesses students through observation, 
students may have difficulty in doing well on written tests and 
vice versa. Perhaps we need to assess with both 
interview/observation and multiple choice items. 

3. Since there was a low correlation between manipulated and 
responding variable tests, perhaps knowing how to identify one 
does not assure students can identify the other. 

The results of this research project indicates that educators 
may not be accurately assesing process skill ability. This may be 
especially relevant when identification of manipulated and 
responding variables are involved. Since there was a low 
correlation between the manipulated and responding variable tests 
and no significant difference between item types, then science 
educators should possibly use more than one type of evaluation 
when testing students on variable identification. With tremendous 
emphasis placed upon standardized test scores, science educators 
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must tenor the results with the Idaa that a paper ?nd pencil test 
may not be the best way t evaluate application level knowledge* 
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A physical education teacher conducted an experiment with her 
class. She found that the students' pulse rate was higher when 
they were active than when resting. 

What IS the responding variable in this experiment? 

A. Amount of activity 

B. Type of activity 

C. Students' pulse rate 

D. Students' weight 



Figure 1. Sample item from the variable identification test 
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RESPONDING VARIABLES 
SELTZER TABLET 


E 


"Here is a glass of water that is at room temperature." 


S 


Given time to feel the glass and/or water. 


£ 


"I am going to drop a seltzer tablet in this glass of water. I want you to 
watch what happens when I drop the seltzer tablet into the water." 


S 


Allowed to watch the dissolving tablet 


E 


"r-^'^re is a glass of hot water ar>d a glass of cold water." 


S 


t 

Given time to feel the glass and/or water. 


E 


"We just dropped a seltzer tablet into the water that is at room temperature. 
Now if we conduct an experiment and drop a seltzer tablet into the glass of 
hot water what things might be different from when the seltzer tablet was 
dropped into the glass of water that is at room temperatu:ie? What would be 
some responding variables? 


S 


Given time to respond. 


E. 


If only one vaiiable is given ask "Are there any more responding variables 
you can think of?" 


E. 


"If ^ye now drop a seltzer table in the glass of cold water what thing might 
be different? What would be some responding variables? 


S 


Given time to respond 


E 


If only one variable is given ask "Are there any more responding variables 
you can think of? 


END 


Figure 2. Script for Responding Variable 
Investigations 
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Table I 

Mean Scores and Standard Deviations 
Post Measures 



Measure 



N 



Maximum Score 



SD 



Range 



Total Variable 
Identification 
Test 

Manipulated 
Subtest 

Responding 
Subtest 

Question 
Format 

Results 
Format 

Hypothesis 
Format 

Description 
Format 

Total 
Investigation 
Score 

Manipulated 
Investigation 
Score 

Responding 
Investigation 
Score 



54 
54 
54 
54 
54 
54 
54 

54 

54 

54 



16 
8 
8 
4 
4 
4 
4 



8.33 
4.24 
4.09 
1.72 
2.46 
2.15 
2.02 

3.93 

2.24 

1.69 



2.78 
1.64 
2.27 
.94 
1.04 
1.04 
1.00 

1.13 

.73 
.75 



3-15 



1-7 



0-8 



0-3 



0-4 



0-4 



0-4 



2-6 



1-3 
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Table 2 



Chart of Correlation Coefficients for All Measures 





T 


M 


R 


Q 


R 


H 


D TI 


MI RI 


T 


















M 


.47* 
















R 


.82* 


-.05 














Q 


.68* 


.41* 


.53* 












R 


.60* 


.23* 


.59* 


.04 










H 


.66* 


.34* 


.60 


.45* 


.31* 








D 


.67* 


.43* 


.47* 


.37* 


.35* 


.13 






TI 


.03 


.06 


.01 


-.01 


-.02 


-.06 


.24* 




MI 


.06 


-.09 


.09 


-.00 


.01 


-.06 


.18 .73* 




RV 


.00 


.15 


-.06 


-.00 


-.04 


-.05 


.20 .71* 


.18 



T = Total Variable Identification Test *significant at .05 level 

M = Manipulated Subtest 
R = Responding Subtest 
Q = Question Format 
R = Results Format 
H = Hypothesis Format 
D = Description Format 
' TI = Total Investigation 
MI = Manipulated Investigation 
RI = Responding Investigation 
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